Endogenous Macrodynamics in Algorithmic Recourse

IEEE — Secure and Trustworthy Machine Learning

Delft University of Technology

Giovan Angela
Aleksander
Karol
Arie van Deursen
Cynthia C. S. Liem

January 16, 2023

Quick Intro

  • Currently 2nd year of PhD in Trustworthy Artificial Intelligence at Delft University of Technology.
  • Working on Counterfactual Explanations and Probabilistic Machine Learning with applications in Finance.
  • Previously, educational background in Economics and Finance and two years at the Bank of England.
  • Enthusiastic about free open-source software, in particular Julia and Quarto.

Motivation and Contributions

In a nutshell …

[…] we run experiments that simulate the application of recourse in practice using various state-of-the-art counterfactual generators and find that all of them induce substantial domain and model shifts.

  • Counterfactual Explanation (CE) explain how inputs into a model need to change for it to produce different outputs.
  • Counterfactual Explanations that involve realistic and actionable changes can be used for the purpose of Algorithmic Recourse (AR) to help individuals who face adverse outcomes.

Figure 1: Dynamics in Algorithmic Recourse: (a) we have a simple linear classifier trained for binary classification where samples from the negative class (\(y=0\)) are marked in blue and samples of the positive class (\(y=1\)) are marked in orange; (b) the implementation of AR for a random subset of individuals leads to a noticable domain shift; (c) as the classifier is retrained we observe a corresponding model shift; (d) as this process is repeated, the decision boundary moves away from the target class.

🔑 Key Contributions

  • We find that the induced shifts are substantial enough to likely impede the applicability of Algorithmic Recourse in some situations.
  • Fortunately, we find various strategies to mitigate these concerns.
  • Our simulation framework for studying recourse dynamics is fast and open-sourced.

Consumer Credit Example

  • Suppose Figure 1 relates to an automated decision-making system used by a retail bank to evaluate credit applicants with respect to their creditworthiness.
  • Assume that the two features are meaningful in the sense that creditworthiness increases in the south-east direction.
  • Then we can think of the outcome in panel (d) as representing a situation where the bank supplies credit to more borrowers (orange), but these borrowers are on average less creditworthy and more of them can be expected to default on their loan.
  • This represents a cost to the retail bank.

Student Admission Example

  • Suppose Figure Figure 1 relates to an automated decision-making system used by a university in its student admission process.
  • Assume that the two features are meaningful in the sense that the likelihood of students completing their degree increases in the south-east direction.
  • Then we can think of the outcome in panel (b) as representing a situation where more students are admitted to university (orange), but they are more likely to fail their degree than students that were admitted in previous years.
  • The university admission committee catches on to this and suspends its efforts to offer Algorithmic Recourse.
  • This represents an opportunity cost to future student applicants, that may have derived utility from being offered recourse.

Background

Algorithmic Recourse

Even though […] interpretability is of great importance and should be pursued, explanations can, in principle, be offered without opening the “black box”. (Wachter, Mittelstadt, and Russell 2017)

::::{.columns}

Framework

Objective originally proposed by Wachter, Mittelstadt, and Russell (2017) is as follows

\[ \min_{x^\prime \in \mathcal{X}} \text{cost}(x^\prime) \ \ \ \mbox{s. t.} \ \ \ M(x^\prime) = y^* \qquad(1)\]

Typically this is approximated through regularization:

\[ x^\prime = \arg \min_{x^\prime} \text{yloss}(M(x^\prime),y^*) + \lambda \text{cost}(x^\prime) \qquad(2)\]

Intuition

Figure 2: A cat performing gradient descent in the feature space à la Wachter, Mittelstadt, and Russell (2017).

Gradient-Based Recourse Revisited

From individual recourse …

\[ \begin{aligned} \mathbf{s}^\prime &= \arg \min_{\mathbf{s}^\prime \in \mathcal{S}} \left\{ {\text{yloss}(M(f(\mathbf{s}^\prime)),y^*)}+ \lambda {\text{cost}(f(\mathbf{s}^\prime)) } \right\} \end{aligned} \qquad(3)\]

  • Here \(\mathbf{s}^\prime=\left\{s_k^\prime\right\}_K\) is a \(K\)-dimensional array of counterfactual states and \(f: \mathcal{S} \mapsto \mathcal{X}\) maps from the counterfactual state space to the feature space.

… towards collective recourse

  • All of the different approaches introduced above tackle the problem of Algorithmic Recourse from the perspective of one single individual.
  • We propose to extend Equation Equation 3 as follows:

\[ \begin{aligned} \mathbf{s}^\prime &= \arg \min_{\mathbf{s}^\prime \in \mathcal{S}} \{ {\text{yloss}(M(f(\mathbf{s}^\prime)),y^*)} \\ &+ \lambda_1 {\text{cost}(f(\mathbf{s}^\prime))} + \lambda_2 {\text{extcost}(f(\mathbf{s}^\prime))} \} \end{aligned} \qquad(4)\]

  • Here \(\text{cost}(f(\mathbf{s}^\prime))\) denotes the proxy for private costs faced by the individual as before and \(\lambda_1\) governs to what extent that private cost ought to be penalized.
  • The newly introduced term \(\text{extcost}(f(\mathbf{s}^\prime))\) is meant to capture and address external costs incurred by the collective of individuals in response to changes in \(\mathbf{s}^\prime\).
  • The underlying concept of private and external costs is borrowed from Economics and well-established in that field: when the decisions or actions by some individual market participant generate external costs, then the market is said to suffer from negative externalities and is considered inefficient (Pindyck and Rubinfeld 2014).

Modeling Endogenous Macrodynamics in Algorithmic Recourse

Research Questions

Does the repeated implementation of recourse provided by state-of-the-art generators lead to shifts in the domain and model?

If so, are these dynamics substantial enough to be considered costly to stakeholders involved in real-world automated decision-making processes?

Do different counterfactual generators yield significantly different outcomes in this context? Furthermore, is there any heterogeneity concerning the chosen classifier and dataset?

What are the drivers of endogenous dynamics in Algorithmic Recourse?

Simulations

Evaluation Metrics

Experiment Setup

\(M\)—Classifiers and Generative Models

tab <- data.frame(
  "Model" = c("MLP","MLP","VAE","VAE"),
  "Data" = c("Synthetic", "Real-World", "Synthetic", "Real-World"),
  "Hidden" = c(32,64,32,32),
  "Latent" = c("-","-",2,8),
  "Layers" = c(1,2,1,1),
  "Batch" = c("-",500,"-","-"),
  "Dropout" = c("-",0.1,"-","-"),
  "Epochs" = c(100,100,100,250)
)
library(kableExtra)
kbl(
  tab, booktabs = TRUE,
  caption = 'Neural network architectures and training parameters.',
  col.names = c("Model","Data","Hidden Dim.","Latent Dim.","Hidden Layers", "Batch", "Dropout", "Epochs")
) |> 
  collapse_rows(1:2, row_group_label_position = 'stack') |>
  kable_styling(latex_options = c("scale_down"))

\(\mathcal{D}\)—Data

Synthetic Data

We use four synthetic binary classification datasets consisting of 1000 samples each: Overlapping, Linearly Separable, Circles and Moons Figure 3.

Figure 3: Synthetic classification datasets used in our experiments. Samples from the negative class (\(y=0\)) are marked in blue while samples of the positive class (\(y=1\)) are marked in orange.

Real-World Data

We use three different real-world datasets from the Finance and Economics domain, all of which are tabular and can be used for binary classification.

  1. The Give Me Some Credit dataset: predict whether a borrower is likely to experience financial difficulties in the next two years (Kaggle 2011).
  2. The UCI defaultCredit dataset (Yeh and Lien 2009): a benchmark dataset that can be used to train binary classifiers to predict the whether credit card clients default on their payment.
  3. The California Housing dataset Pace and Barry (1997): continuous outcome variable binarized as \(\tilde{y}=\mathbb{I}_{y>\text{median}(Y)}\) indicating if the median house price of a given district is above the median of all districts.

\(G\)—Generators

Experiments

Mitigation Strategies and Experiments

Discussion

Limitations and Future Work

Concluding Remarks

Questions & Answers ❓

Final Things 🏁

More Resources 📚

Read on …

  • Related blog posts (hosted on this website that itself is built with Quarto and involves lots of Julia content): [1] and [2].
  • Blog post introducing CE: [TDS], [blog].
  • Blog post on Laplace Redux: [TDS], [blog].
  • Blog post on Conformal Prediction: [TDS], [blog].

… or get involved! 🤗

Image Sources

  • Quarto logo. Source: Quarto
  • Julia to Quarto animation. Source: author (heavily borrowing from Javis.jl tutorial)

References

Antorán, Javier, Umang Bhatt, Tameem Adel, Adrian Weller, and José Miguel Hernández-Lobato. 2020. “Getting a Clue: A Method for Explaining Uncertainty Estimates.” https://arxiv.org/abs/2006.06848.
Joshi, Shalmali, Oluwasanmi Koyejo, Warut Vijitbenjaronk, Been Kim, and Joydeep Ghosh. 2019. “Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Making Systems.” https://arxiv.org/abs/1907.09615.
Kaggle. 2011. “Give Me Some Credit, Improve on the State of the Art in Credit Scoring by Predicting the Probability That Somebody Will Experience Financial Distress in the Next Two Years.” Kaggle. https://www.kaggle.com/c/GiveMeSomeCredit.
Mothilal, Ramaravind K, Amit Sharma, and Chenhao Tan. 2020. “Explaining Machine Learning Classifiers Through Diverse Counterfactual Explanations.” In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 607–17.
Pace, R Kelley, and Ronald Barry. 1997. “Sparse Spatial Autoregressions.” Statistics & Probability Letters 33 (3): 291–97.
Pedregosa, Fabian, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, et al. 2011. “Scikit-Learn: Machine Learning in Python.” The Journal of Machine Learning Research 12: 2825–30.
Pindyck, Robert S, and Daniel L Rubinfeld. 2014. Microeconomics. Pearson Education.
Schut, Lisa, Oscar Key, Rory Mc Grath, Luca Costabello, Bogdan Sacaleanu, Yarin Gal, et al. 2021. “Generating Interpretable Counterfactual Explanations By Implicit Minimisation of Epistemic and Aleatoric Uncertainties.” In International Conference on Artificial Intelligence and Statistics, 1756–64. PMLR.
Wachter, Sandra, Brent Mittelstadt, and Chris Russell. 2017. “Counterfactual Explanations Without Opening the Black Box: Automated Decisions and the GDPR.” Harv. JL & Tech. 31: 841.
Yeh, I-Cheng, and Che-hui Lien. 2009. “The Comparisons of Data Mining Techniques for the Predictive Accuracy of Probability of Default of Credit Card Clients.” Expert Systems with Applications 36 (2): 2473–80.